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Abstract 

Studies focusing on the effectiveness of computer-assisted instruction have been 
growing recently in Turkey. In this research, quantitative studies comparing the ef- 
fectiveness of computer- assisted instruction to traditional teaching method and con- 
ducted between 1998 and 2007 are studied by meta analysis. Seventy eight studies 
that have eligible data were combined with meta analytical methods by coding pro- 
tocol from the 422 master’s and doctoral degree and 124 articles. As a result for the 
study, the effect size of computer-assisted instruction method for academic achieve- 
ment calculated 1.048. This is large scale according to Thalheimer and Cook, large 
and Cohen, Welkowitz and Ewen (2000). Recommendations were made based on 
the results of the study. 
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The effectiveness of CAI as compared to traditional instruction has 
been investigated for decades. Various studies have been conduc- 
ted at different educational levels and in various areas of instructi- 
on (Demirer, 2006; Genftiirk, 2003; Katircioglu and Kazanci, 2003; 
Tutaysalgir, 2006; Zhou, Brouwer, Nocente and Martin, 2005). Nu- 
merous studies have investigated the effectiveness of computer-as- 
sisted instruction among different samples (Chang, 2002; Hacker 
and Sova, 1998; Hollingsworth and Hoover, 2001; Jacoby, 2005; 
Wong, 2001). 

Both methods have unique features but either one may be used to 
provide instruction for a particular learning experience. This raises 
the question of which is the more effective method for a better le- 
arning experience. One way to answer this question would be to 
compare the two methods by measuring the respective gains in le- 
arning. Hence, there have been numerous studies of computerized 
instruction’s effectiveness (Yaakub, 1998). 

There is a need for a meta-analytic effect analysis of research carri- 
ed out in Turkey in order to reveal the effectiveness of computer- 
assisted instruction from a broader point of view. Therefore, the 
aim of this paper is to bring the bigger picture to light by calcula- 
ting the effect size of academic success in lessons where computer 
assisted instruction is used. In order to analyze the impact of com- 
puter-assisted instruction 78 research studies were identified. The 
research question that guided the analysis was ‘Does computer- as- 
sisted instruction have a positive impact on learners’ academic suc- 
cess when compared to traditional teaching method?’ 


Methods 

Meta-analytic effect analysis was used in order to calculate the ef- 
fect size of computer-assisted instruction on academic success. Me- 
ta-analysis can be defined as an analysis of previous analyses and is 
a statistical method used to combine the results of similar studies 
(Lyons, n.d; Ergene, 2003). 

Data Collection 

For meta-analysis, master’s theses, doctoral dissertations, published 
articles, electronic catalogues of university libraries, and symposi- 
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um proceedings were used as sources of data. Initially, Turkish and 
English keywords were used to search online thesis centre of ‘The 
Council of Higher Education National Thesis Center.’ The search 
using keywords such as ‘bilgisayar destekli egitim’, ‘ bilgisayar destek- 
li ogreUm’, ‘bilgisayar destekli ogrenme’, ‘computer based learning \ 
‘computer- assisted teaching \ ‘computer- based training ’ revealed a 
total of 378 studies composed of 155 master’s theses and 63 docto- 
ral dissertations. As a result, the sample for the research was finali- 
zed and 34 studies on the effectiveness of computer- assisted ins- 
truction were included in meta-analysis. 

Inclusion Criteria 

The following criteria were used for the selection of studies to be 
included in the current research (§ahin, 2005, 33): 

Criterion 1: Time interval of the studies to be included in meta- 
analysis. 

Studies must be conducted in the last decade (1998-2007). 

Criterion 2: Published and unpublished resources. Master’s theses 
and doctoral dissertations, periodical academic journals, electronic- 
academic journals, databases, proceedings of conferences and 
symposia. 

Criterion 3: Appropriateness of the research design used in the stu- 
dies. 

Criterion 4: The use of instructional method that fits the research 
aim. 

Criterion 5: To have sufficient numerical data. 

Only studies that provide the following values for the groups were 
included the meta-analysis: sample size, mean and standard devi- 
ation. 

Coding Method 

After the studies that meet the inclusion criteria are identified, a co- 
ding method is required to be used for the comparisons to be conduc- 
ted in the next stage. The coding method should be general enough 
to allow inclusion of all studies and at the same time specific enough 
to allow differentiation (Ozcan, 2008). The coding method for the 
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current study was composed of three main sections and a total of 12 
questions. The first section was called ‘ study identity ’. This section 
had 7 questions. In order to define the study identity, information 
such as the number of study, the name of study, author names, the 
year and place of study and publisher name were specified. The se- 
cond section was called ‘ study contents’ and had 4 questions (Ozcan, 
2008). Information regarding the name of the lesson and year of scho- 
ol where computer- assisted instruction was used, the length of ins- 
truction, criteria for the selection of participants and the details of the 
computer- assisted instruction being used was gathered. The third 
and last section was ‘ study data ’. Descriptive statistical data such as 
sample size, mean values and standard deviation values of the expe- 
rimental and control groups were specified in this section. 

Study Characteristics 

The study characteristics are the independent variables of the me- 
ta-analysis (Tarim, 2003). The study characteristics for the current 
study were identified as follows: education levels of the partici- 
pants, lessons in which the studies were conducted, places where 
the studies were conducted, sample volumes of the studies, stan- 
dard deviation of the samples in the studies, mean values of the 
samples in the studies. 

Data Analysis 

The current study uses ‘ study effect’ meta-analysis for the analysis 
of the data. This method is used for group differences when the 
arithmetical mean values of the dependent variables of each study 
included in meta-analysis were not obtained using the same scale 
(Lipsey and Wilson, 2001; Cohen, 1988 cited in Ozcan, 2008). The 
aim of this method is to calculate the difference between the mean 
values of the control and experimental groups in experimental stu- 
dies represented by the formula d= (Xe-Xc)/SD (Hunter and 
Schmidt, 1990). The “d” value obtained represents the effect size 
and forms the basis for meta-analysis. On the other hand, in order 
to carry out the calculations for meta-analysis, CMA statistics pac- 
kage was used as the basis, as well as NCSS and Microsoft Excel 
2003 programs. 
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Results 

Considering the 78 studies which provide information on the samp- 
le size, the experimental group consisted of 2536 and the control 
group consisted of 2560 students. The studies that involve the hig- 
hest amount of data were conducted in the year 2006, in Ankara, at 
the second stage of the primary school and in science lessons. 

The results of the meta-analysis conducted based on fixed effects 
model indicates that academic success was higher in computer- as- 
sisted instruction than in traditional teaching method; the standard 
deviation was 0.03; upper limit of 95% confidence interval was 1.01 
and lower limit was 0.89 and the mean effect size (Point estimate) 
was ES= 0.95. The mean effect size calculated as 0.95 was accepted 
to be large in the classifications of both Thalheimer and Cook (n.d.) 
and Cohen et. al. (2000). Z test calculations revealed statistically 
significant at .01 level (z= 31.81; p<0.01). 

At the end of the homogeneity test, Q statistical value was calcula- 
ted to be 424.95. As found on the table, 77 degrees of freedom 
at 95% significance level was 90.53. Q statistical value of 424.95 was 
found to exceed the critical value of chi-square distribution at 77 
degrees of freedom (X^ 095= 90.53). These values indicated that 
the distribution of the effect sizes of the studies were heterogeneo- 
us in terms of fixed effects model. The studies being heterogeneo- 
us as indicated by the Q statistical value means that effect size va- 
riance is bigger than the variance that could be expected as a result 
of sampling error (Ozcan, 2008). It is also means that it is not pos- 
sible to argue the assumption of fixed effects model, i.e. there be- 
ing a single real effect that can be anticipated by the findings of the 
studies. As a result, through analyses based on random effects mo- 
del, illusions caused by the heterogeneous sample can be elimina- 
ted (Yildiz, 2002; Demirel, 2005). Following this, the effectiveness 
of traditional teaching method and computer- assisted instruction 
were compared using random effects model. 

The results of the analyses conducted according to random effects 
model indicate that computer- assisted instruction is more effecti- 
ve than the traditional teaching method in terms of academic suc- 
cess with a standard deviation of 0.07; a upper limit of 1. 19 and a lo- 
wer limit of 0.91 for 95% confidence interval and a mean effect si- 
ze of ES=1.05. The effect size value obtained (d=1.05) is conside- 
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red to have a huge effect in terms of the classification by Thalhei- 
mer and Cook (n.d.), and a large effect in terms of the classification 
by Cohen et. al. (2000). Z test calculations revealed statistically sig- 
nificant at .01 level (z= 14.56; p<0.01). 


Discussion 

In this study, the mean effect sizes of the 78 studies that were inc- 
luded in meta-analysis in terms of the two effects models used for 
statistical analysis. The effect of computer- assisted instruction on 
academic success had a value of 0.95 in terms of fixed effects mo- 
del which meant that academic success was higher in computer- as- 
sisted instruction. Similarly, in terms of random effects model aca- 
demic success was again higher in computer- assisted instruction 
with a value of 1.05. This value indicated a large, positive and sig- 
nificant effect size according to Cohen et. al. (2000). 

The results of the meta-analysis suggested that computer- assisted 
instruction led to greater academic success than traditional teaching 
method. This finding is consistent with the results of other national 
and international studies (e.g., Ak§ay, Feyzioglu, and Tiiystiz, 
2003; (2ekba§, Yakar, Yildirim and Savran, 2003; Demirer, 2006; 
Gen^tiirk, 2003; Katircioglu and Kazanci, 2003; Kiise, Ayas and Ta§, 
2003; Sagner, Pheleps and Fienhold, 2000; Tezcan and Yilmaz, 
2003; Tosun, 2006; Tutaysalgir, 2006; White and Bodner, 
2001;Wong, 2001; Zhou, Wytze and Brouwer, 2005). Current fin- 
dings are also consistent with the results of the meta-analysis study 
on the effectiveness of computer- assisted instruction conducted by 
the researchers (Bangert-Drowns, 1985; Hasselbring, 1984; Kulik, 
1983; Kulik and Kulik, 1985). 

Kulik (1983) observed that at least a dozen meta-analyses on CAI ef- 
fectiveness have been performed. He reviewed these and discovered 
the following points: Students like their classes more when taught 
through CAI. CAI students learn in less time as compared to students 
taught in the traditional way. Consequently, students learn more in 
CAI classes as compared to traditional instruction (Yaakub, 1998). 
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